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Similar to their colleagues from other countries, French academic librarians are involved in the 
promotion of data management plans and FAIR (Findable, Accessible, Interoperable, Reusable) 
principles, as first steps towards more transparent research. But reproducible research requires one to 
dive into a wide range of subjects (e.g.: statistics, study design, scientific computing) that are quite far 


from a French librarian’s daily practices and/or initial training given that most have a degree in 
literature or history (1). Librarians do have technical expertise, but horror stories (2) pinpoint the fact 


that even seasoned academics and data science experts seem to lose track of current techniques, after 
decades of dizzying technical and methodological mutations. In such a context, what could be the 
added value of librarians? To what extent do they have to plan a mutation of their skill sets? See 


summary in Table 1. 


Classy but trendy: information skills at the 
forefront of reproducible research 

Statistical and computational skills are key 
components (3-6) of more reproducible 
research. However, is there room for people 
who are not familiar with the command line or 
statistics methods? Of course, some librarians 
have developed data management skills. 
Nevertheless, the stakes of traceability also 
bring back to the forefront a librarian’s 
traditional skills. In more fundamental 
disciplines, the reproducibility of research is 
seen less as the ability to obtain similar results 
than as the capacity to track back the 
argumentation. In this perspective, the 
management of references and more 
particularly the follow-up of retracted articles 
play a major role. Data management skills thus 
remain central, especially when it comes to 
taking advantage of tools that automate tasks, 
such as Zotero. 

Indeed, many services focusing on information 
needs can be targeted in the reproducibility 
perspective. Practices such as systematic 
review (7) can also help to limit the impact of 
biases (e.g.: selective reporting, confirmation 
bias). Information addressing a_ specific 
research question is exhaustively collected by 
using pre-established and __ standardized 
eligibility criteria (eg.: type of method 
applied, source of funding, measurements 
obtained, etc.). The data are then synthesized 
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with or without the use of statistical methods. 
Systematic reviews are common in health 
sciences (8) and through adaptations, their use 
could be extended to other fields as they can 
limit the impact of bias toward positive 
outcomes (9): the published results represent 
only part of a corpus composed of different 
types of data, including grey literature (e.g.: 
reports, unpublished research outputs, 
technical reports, etc.) . 

The standardization of information-gathering 
methods and a broadening of documentary 
sources contribute to the transparency of 
research. However, not all editorial forms 
benefit from the same visibility due to research 
evaluation criteria and the policies of 
traditional publishers who value the article in 
its canonical form. Thus, replication studies 
(10-12) or forms of pre-registration (13-15) 
[i.e.: registered reports, registered replication 
reports (16)] deserve greater publicity. For 
librarians, it doesn’t mean to promote these 
sources without nuance as pre-registration is a 
subject of debate (17), but to synthesize the ins 
and outs, and to present the platforms for 
where to find them (e.g.: Open Science 
Framework). In their field, librarians can thus 
participate in limiting the "file drawer effect" 
(5,9,18). The file drawer effect can be defined 
as a bias that leads researchers to put in the 
“drawer” anything that doesn’t fit into the 
criteria of traditional editors who favor 


positive and catchy results: “Modern science’s 
professional culture prizes positive results far 
above negative results, and also far above 
attempts to reproduce earlier 
research. Scientists therefore steer away from 
replication studies, and their negative results 
go into the file drawer.” (5) 


Decathlete fatigue: challenges of skill 
specialization 

Beyond interdisciplinary stakes, the advent of 
data-driven research and the issues of research 
reproducibility have accentuated the need for a 
plurality of expertise in teams. Of course, 
having a basic knowledge of computer science 
and/or statistics is of major interest, but the 
level of expertise to be achieved is such that 
the competency should no _ longer be 
understood at the level of an individual, but at 
the level of a whole team, according to 
Wessel (19): “Statistical illiteracy among 
researchers is one of the reasons why 
psychology as a whole is in trouble. The 
solution is thought to lie in educating people 
about statistics. Of course, furthering a basic 
understanding is important, but investing in 
statistical skills at an individual level is not the 
ultimate solution. There is a limit to what 
individual researchers can do.” Leek (20) also 
emphasizes the need for specialists to ensure 
appropriate use of statistical methods: “Since 
most people performing data analysis are not 
statisticians there is a lot of room for error in 
the application of statistical methods. This 
error is magnified enormously when naive 
analysts are given too many ‘researcher 
degrees of freedom’.” Moreover, even the 
choice of the method to be applied is arduous 
for a non-specialist according to Wilcox and 
Rousselet (21): “But for the non-statistician, 
the vast array of new and improved techniques 
for comparing groups and __ studying 
associations can seem daunting, simply 
because there are so many new methods that 
are now available.” 

At the same time, thanks to adapted graphical 
user interfaces, tools that can be used without 
advanced technical skills have been developed. 
But the use of these tools doesn’t solve the 
problem of the reliability of the results. 
According to Plesser (22), methodological 
questions are obliterated because they are 
hastily delegated to tools: “Since digital 
computers are exact machines, practitioners 
apparently assumed that results obtained by 
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computer could be trusted, provided that the 
principal algorithms and methods employed 
were suitable to the problem at hand. Little 
attention was paid to the correctness of 
implementation, potential for error, or 
variation introduced by system soft- and 
hardware [...].” In addition, these tools, which 
are suitable for non-specialists, are still part of 
broader and more complex ecosystems, the full 
control of which requires an advanced 
technical background. 


A patchwork is not a canvas: disconnected 
expertises 

Bringing different expertises together responds 
to immediate functional needs and also allows 
for a richer approach to the research question, 
according to Munafo and his co-authors (14): 
“This also brings greater scope for multiple 
theoretical and disciplinary perspectives, and a 
diverse range of research cultures and 
experiences, to be incorporated into a research 
project.” Yet, as Bammer (23), Brown (24), 
and Jang (25) consider, the juxtaposition of 
skills doesn't always ensure better results 
because it can also be a vector of dissonance.: 
“T...] cultural differences between team 
members can lead to inconsistent norms and 
conflicting assumptions, making it difficult to 
access and synthesize the diverse knowledge 
they hold.” 

If left unmanaged, interdisciplinary and 
inter-professional collaboration can increase 
the risk of error, as Yarborough and co-authors 
note (26): “Indeed these errors [study design 
and data analysis errors] are likely to increase 
due to trends in current scientific practice, 
particularly the growing size and 
interdisciplinarity of investigative teams.” 
Non-reproducibility has both technical and 
methodological causes and stems from 
organizational and behavioral issues. Increased 
segmentation of work between experts as well 
as the reinforcement of the dynamic of 
specialization do not lead to more reliable 
research. Leonelli (27) considers it to be an 
aggravating factor. When a plurality of actors 
are involved, a fortiori during the complex 
process of analysis, clearly defining 
responsibilities is challenging: “[A plurality of 
actors results in] a general increase in the 
complexity and scale of experiments and 
related statistical analyses of results, with 
increasing specialization and division of labor 
resulting in lack of clarity around who is 


actually responsible for quality checks.” 
Yarborough and his co-authors (26) also point 
out the increased risk of errors resulting from 
the division of work and expertises: “[...] 
Because they require divisions of labor and 
expertise, such collaborations create fertile 
ground of producing unreliable research.” 

As noted by Deardorff (28), the distribution of 
expertise engages researchers to be 
accountable for steps carried out by others: 
“For some researchers, this meant taking 
ownership of a step that had previously been 
done by a collaborator”. But even more than in 
the past, identifying the parameters that impact 
the results can be tricky (4), even at the team 
level: “When closely scrutinized, a scientific 
study or experiment may be seen to entail 
hundreds or thousands of choices, many of 
which are barely conscious or taken for 
granted. [...] An investigator may or may not 
realize when a possible variation could be 
consequential to the replicability of results.” 
Moreover, the issues of — research 
reproducibility accentuate the need for a 
reinforced dialogue as they question what 
appears to be self-evident: what is a result? 
What are we trying to reproduce: a model, a 
process, an object? The variety of meanings of 
the phrase "reproducible research" as analyzed 
by Barba (29) emphasizes the need for 
contextualization. Therefore uncovering the 
cultural gaps inherent in professional and 
disciplinary practices isn’t considered by the 
Committee on Reproducibility and 
Replicability in Science (4) as a futile effort: 
“Despite efforts to coalesce around the use of 
these terms, lack of consensus persists across 
disciplines. The resulting confusion is an 
obstacle in moving forward to improve 
reproducibility and replicability.” 


The process of "cultural brokerage" at the 
service of interdisciplinarity? 

The coexistence of different types of expertise 
can lead to an increased risk of errors insofar 
as the actors have their own norms, beliefs and 
hypotheses. Thus, Brown (24) mentions 
“substantial transaction costs” that come with 
interdisciplinarity. Of course, other parameters 
also affect the dynamics of a team (e.g.: the 
inadequacy of evaluation criteria, the 
legitimacy that each actor grants himself 
according to their place in the academic 
community, the stakes inherent in a career 
stage). But this contribution focuses on the 
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issue of cultural heterogeneity within a group 
and the possible role of librarians. 

According to Jang (25), a process of "cultural 
brokerage" can foster cohesion within a team: 
“Brokerage is typically depicted as taking 
place between actors who have no access to 
one another—for instance, at the intersection 
of organizational or professional boundaries.” 
Thus, cultural brokerage is not about dividing 
tasks or directly transferring skills between 
actors, but about overtaking the hiatus in 
professional cultures. Jang distinguishes 
"cultural insiders", who share all or part of a 
team's knowledge, from "cultural outsiders", 
whose knowledge doesn’t overlap with that of 
the group. Insiders act as translators (i.e. 
“resolving”); they are messengers (30). 
Outsiders connect team members by involving 
them in the process of capturing the group's 
tacit norms (i.e.: facilitating) (30); they don't 
directly provide an answer. Moreover, while 
cultural insiders may informally take on an 
intermediation role, outsiders need their 
assignment to be formally posed within the 
team to be able to fully play their role. 


From “embedded librarian” to “cultural 
broker” 

Because of their role in the academic 
community and the nature of their skills, 
librarians seem predisposed to engage in the 
process of cultural brokerage. Indeed, the well 
documented (31) concept of the "embedded 
librarian" already refers to the notion of an 
information professional integrated into a 
group of users, in a punctual or more perennial 
way, online or in a physical way. This consists 
of an in-depth and personalized documentary 
assistance, resulting from empathetic 
capacities and knowledge of the treated topic. 
The concept of "embedded librarian" isn't new, 
but this role can evolve towards that of a 
"cultural broker". 

Should librarians aim to become cultural 
insiders? Not necessarily, because the 
contribution of outsiders is equal or even 
superior to that of insiders (25,30). This 
finding is surprising because it’s precisely the 
perception of their expertise that penalizes the 
insiders. Indeed, outsiders have the advantage 
of being viewed as neutral third parties, less 
inclined to judge than insiders. In addition, 
given their greater proximity to group 
members, the insiders' clarification efforts may 
be perceived as unnecessary. Added to this, the 


inclusion of outsiders would secure the group's 
cohesion in the long term (30): “Cultural 
outsiders may be particularly valuable in teams 
with long time horizons, where team viability 
is crucial.” Indeed, outsiders educate teams to 
enact internally the cultural brokerage process. 


Toward a “blameless post mortem culture”: 
enabling supportive work environments 
Data science standards have been drastically 
raised under the pressure of reproducibility 
challenges, but the acquisition of technical 
knowledge is not only a technical issue. The 
point is less to seek to achieve the level of 
expertise to deal with the issue than 
developing dialogue skills. Training eases the 
dialogue between researchers with different 
technical levels, according to Deardoff (28): 
“[...] the workshop had helped demystify a 
complicated topic. [...] [Researchers] felt more 
comfortable talking about programming with 
their collaborators [...].’In addition to 
traditional training, librarians can create or 
support communities of practice such as 
"ReproduciliTea" (32), “The Turing Way” 
(33). Activities can be designed by type of 
needs, not just by level of technicality: for 
example, data engineers' concerns are more 
focused on infrastructure and IT equipment. 
Data analysts and machine learning engineers 
tend to focus on front-end issues (i.e.: analysis 
and presentation of results). A “Reprohack” is 
a relevant way to favor dialogue skills (34). 
Their expertise in scientific monitoring and 
information retrieval will help librarians to 
identify experts who can conduct these 
activities. 

Actions based on collaborative practices (e.g.: 
hackathons, peer programming sessions, code 
reviews, sprints), contribute to initiating a 
broader dialogue (35-37). For instance, by 
easing capturing and _ codifying tacit 
knowledge, a book sprint (38-41) enables 
cultural brokerage. In a couple of days, a group 
of authors supported by a facilitator writes a 
complete book from scratch [e.g.: book sprints 
organized by librarians (42,43)]. According to 
Baker and her co-authors (38), “[...] the book 
sprinting process, by _ stressing the 
collaborative aspects, materializes the 
being-together, working-in-tandem, sharing 
and intersubjective exchange of ideas, but also 
written text, submerging authorship into a 
collective form of writing which is extremely 
powerful”. If the group of authors is 
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homogeneous, the book sprint helps them to 
describe and synthesize their knowledge and 
the role of the facilitator is to introduce 
coherence (38) (i.e.: “extractive mode”). In 
contrast, when group members have little in 
common, the facilitator focuses on the 
emergence of new concepts and helps authors 
to put some distance from their existing 
practices (i.e.: “generative mode”). However, 
without prior analysis of the context, 
collaborative activities turn out to be 
counterproductive: a hackathon turns into an 
unhealthy competition, the pair programming 
session becomes an evaluation. Once again, 
neither the bringing together of actors nor the 
application of a collaborative method suffices 
to trigger the process of cultural brokerage and 
the added value of librarians lies precisely in 
their knowledge of the practices and norms of 
"academic tribes" (44). 

The skills gap plays a role in group 
interactions but the problem has first a 
behavioral nature, as Varoquaux points out 
(45): “Inability to understand the others’ point 
of view, or to communicate ours, can bring in 
emotions. Emotions most often impedes 
technical communication”. But the emergence 
of a new ethos is a long term goal, hampered 
by academic culture as Herterich and her 
co-authors (46) point out: “Academics are 
pressured by this academic fallacy that they 
need to be brilliant at all aspects of the job, 
which means they are reluctant to share 
something they’re not brilliant at.” Therefore, 
enabling safe working environments should be 
a priority. It’s neither a luxury nor a fashion, 
and therefore Herterich and her co-authors (46) 
put psychological barriers and material 
constraints on the same level : “limitations on 
time, on resources, and on confidence”. 
Bishop (47) highlights the cultural change 
inherent in more open and _ transparent 
research: “As open science becomes 
increasingly the norm, we will find that 
everyone is fallible.’ Therefore Parker (48) 
calls for a “blameless post mortem culture”. In 
this context, being seen as neutral third parties 
seems to be an advantage for librarians. 


Conclusion 

Reproducible research mobilizes a set of 
methods and techniques that are a priori far 
removed from the field of librarians. However, 
the dynamic of hyper-specialization that has 
been at work for several years now shows the 


need for a form of cultural brokerage because 
the juxtaposition of expertise can accentuate 
the risk of errors. While librarians can place 
their more traditional skills in the broader 
context of reproducibility, they also have the 
opportunity to strengthen their connections 
with their users by moving from embedded 
librarian to cultural broker. The development 
of skills, especially computational skills, plays 
a major role, but not only on a technical level: 


Table 1. 


Challenges encountered by researchers, 
linked to the reproducibility of research 
Publication bias favor positive results 


Emergent editorial forms may suffer from a | Getting familiar with these emergent 
editorial forms. 


lack of visibility among researchers 


Tracking retracted articles requires 
additional attention 


Standards in data science required by 


expertise needs to be distributed among a 
wide pool of actors. 

But the juxtaposition of expertise also 
increases the risk of errors. 


Broadening the diversity of sources . 


Revamping source workflow of 
bibliographic records. 


Capturing tacit norms, beliefs and 
reproducible research needs are so high that | hypotheses from different stakeholders 


in order to lower the cost of these 
disconnected expertises. 
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it’s also required for building consistency 
between stakeholders. Indeed, changes in 
organizational behavior call for deep cultural 
change. Therefore librarians can take 
advantage of the fact that they are seen as 
neutral agents by researchers. But they first 
need to be identified as potential collaborators. 
Thus, the question of visibility remains a major 
issue for librarians. 


Need How librarians may support 


researchers 
Standardizing information 
gathering. 


Summarizing the ins and outs 
of emergent editorial forms to 
feed the debate. 

Selecting platforms where this 
material is available. 

Training in the use of tools that 
automate the management of 
sources (ex: Zotero). 

Training in the methods. 
Playing the role of “cultural 
broker”: offering and/or 
supporting collaborative 
environments for researchers. 
As librarians may be seen as 
neutral third parties, they may 


Transparency calls for disclosure, but 
cultural academic norms prevents 
researchers from moving towards a new 
ethos 


Building inclusive and secure work 
spaces, developing soft skills that help 
nurture hard skills: expertise and 
performance may be threatened by a 
lack of ad hoc soft skills. 





play a positive role in the 
dissemination of methods and 
topics. 


Table 1: a non-exhaustive list of librarians’ set of skills that could be relevant to address reproducible research challenges. 
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